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EXECUTIVE SUMMARY 

Introduction 

In 1998, the Missouri Office of the State Courts Administrator (OSCA) developed a 
comprehensive juvenile offender classification system that included a risk assessment, a 
classification matrix based upon risk and most serious offense, and a needs assessment. The 
classification system was implemented in 11 counties and the city of St. Louis in March of 
1999.' In July of 1999, a software product called J-TRAC, developed by the Office of State 
Courts Administrator to automate the classification system, was released to all of the 
participating implementation circuits. In December 2000, OSCA contracted with the National 
Council on Crime and Delinquency (NCCD) to conduct a second validation study of the risk 


assessment scale. 


Methodology 

The analysis presented in this report had two goals: 1) to assess the performance of the 
original risk assessment, and 2) to determine if changes to the risk assessment would improve its 
classification accuracy. To determine how effectively the existing risk assessment worked, 
NCCD examined the relationship between the existing risk classification and subsequent 
delinquency referrals. Subsequent analyses was conducted to determine whether or not 
alterations to the existing assessment could improve performance. These analyses included 
evaluating the existing risk items and cut points and developing an actuarial risk assessment 


using available data. These analyses involved both bivariate and multi-variate statistical 


' Circuits 10, 11, 12, 19, 20, and 22 implemented the classification system. 
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techniques to examine the relationship between the risk items and the outcomes.” Data were 
analyzed for the total sample as well as subgroups defined by youths’ ethnicity, gender, and 
geographic location. 

The primary outcomes analyzed were subsequent referral (of any type) and adjudications. 
Subsequent referral was selected as a primary outcome measure because the 1998 validation 
study conducted by Dr. Leonard used this measure and retaining it provides a method of 
comparison. Subsequent adjudication was considered an important measure because court 
involvement indicates substantial evidence that the youth committed the alleged offense, and 


adjudication rates are less disparate across key subgroups. 


Findings 

An effective and valid risk assessment has progressively higher recidivism rates that 
correspond to each increase in risk classification level across multiple outcomes. Ideally, the 
rates between consecutive risk levels maximize the separation between the high and low risk 
groups, as well as between consecutive risk groups. The best way to assess the performance of 
the risk assessment versions, then, is to compare the separation between risk levels between the 
high and low risk groups and between consecutive groups. 

The risk assessment currently in use by officers in participating circuits provided risk 
classifications that identify youth with significantly different rates of subsequent delinquency for 
most outcomes. However, analysis of the risk items and the cut points that define the risk 


classifications suggested that certain modifications could improve the classification accuracy of 





> A variety of statistical methods could be used to conduct the analyses described. A prior study by Simon (1971) and an 
exhaustive study by Gottfredson and Gottfredson (1979), later confirmed by other researchers (see Wilbanks, 1985 and Benda, 
1987), found that less precise methods of statistical evaluation (including bivariate analyses or least squares regression) often 
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the assessment. In addition, a full-scale re-examination of the relationships between the risk 
items officers observed at the time of the youth’s referral and subsequent delinquency resulted in 
an alternative risk assessment that differed from the current one. The risk assessment versions 
presented in this report are: 

1. The original risk assessment: the risk assessment currently used by Missouri 
officers. 

2. The original risk assessment with revised cut points: the original risk assessment 
with no changes to the items but the revised cut points are defined as low risk, 0-4; 
moderate risk, 5-16; and high risk, 17 points or more. 

3. The original risk assessment with re-weighted items: the original risk assessment 
with item weights reduced for all risk items, and cut points altered relative to the new 
distribution of risk scores. To clarify, this version retains all risk items and choices 
per item on the original risk assessment, but changes the number of points assigned 
when an item is found true. These changes are reviewed in Appendix A. 

4. The re-developed risk assessment: the risk assessment that resulted from a complete 
multi-variate analysis of the data. This assessment includes many of the re-weighted 
items that are present in the previous version of the assessment, but includes 
additional changes. These additional changes are: a) replacing the current assault 
referral item to a yes/no item (dropping the separation between felony and 
misdemeanor assault referrals); b) collapsing moderate and severe school problems 
into one category for the school behavior item; c) similarly collapsing the parental 
management item; and d) eliminating the parents’ criminality item. This assessment 
is also shown in Appendix A. 

These versions of the risk assessment, as expected, resulted in slightly different 
distributions by risk level (see Table El). When applied to all sampled youth, the original risk 
assessment classified 49.1% of the youth as low risk, 30.5% as moderate risk and 20.4% as high 
risk. In comparison, the other risk assessment versions classified a higher proportion of the 


sampled youth as moderate risk. The original risk assessment with revised cut points classified 


57.2% of youth as moderate risk and 10.7% as high risk. The original risk assessment with 


produce the best overall result. These procedures were employed in this analysis. 
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revised cut points and re-weighted items similarly classified 62.8% of youth as moderate risk and 
11.3% as high risk. The re-developed assessment classified 55.2% of the youth as moderate risk 


and 18.1% as high risk. 








Table E1 


Risk Level Distribution of Youth 
Classified by Risk Assessment Version 














Low Risk Moderate Risk High Risk Total 
N % N % N % N % 
1. Original Risk Assessment 1,429 | 49.1% 888 30.5% 594 20.4% 2,911 100.0% 





2. Original Risk Assessment with 


0 0 0 0 
Ravised CupPoinis 935 | 32.1% 1,664 | 57.2% 312 10.7% | 2,911 | 100.0% 





3. Original Risk Assessment with 


Re-Weighted Items? 753 | 25.9% 1,829 | 62.8% 329 11.3% | 2,911 | 100.0% 






































4. Re-Developed Risk Assessment | 776 26.7% 1,608 55.2% 527 18.1% | 2,911 100.0% 








Table E2 reviews the primary outcomes for each version of the risk assessment for the 
overall sample. The differences in re-referral and subsequent adjudication rates among youth 
classified as low risk versus high risk were greater for the altered versions of the risk assessment 
than for the original risk assessment. 

For example, when the original risk assessment was applied, the re-referral rate for youth 
classified as high risk was 1.6 times greater than the rate for low risk youth (54.5% and 20.6%, 
respectively).’ When the original risk assessment with revised cut points is applied, youth 
classified as low risk had a 15.7% re-referral rate, while 59.6% of high risk youth had a 


subsequent referral (2.8 times greater than that of low risk youth). 


> This version also has, by necessity, revised cut points. For ease of reference, however, future tables will use the label “the 
original risk assessment with re-weighted items.” 


* This comparison (a percentage increase) is calculated by dividing the difference in rates by the lower rate. For example, the 
(high risk rate — low risk rate) is divided by the low risk rate. The purpose of this comparison is to enable comparisons of 
differences, while controlling for the lower rate. For example, the difference between low-moderate risk and moderate-high risk 
might both be 10%, but the relative increase would be very different. 
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When applying the original risk assessment with revised cut points and re-weighted 


items, 13.1% of youth classified as low risk and 60.2% of youth classified as high risk had a 


subsequent referral (3.6 times greater than that of low risk youth). Youth classified as low risk 


by applying the re-developed tool had a re-referral rate of 13.4%, while high risk youth had a re- 


referral rate of 57.9% (3.3 times greater than the rate for low risk youth). 
















































































Table E2 
Findings 
by Risk Assessment Version 
% with 
Subsequent % with Subsequent 
Total N Referral Adjudication 

Total Sample 2,911 34.1% 8.6% 
1. Original Risk Assessment 

Low Risk 1,429 20.6% 4.3% 

Moderate Risk 888 42.2% 11.3% 

High Risk 594 54.5% 14.6% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935, 15.7% 3.0% 

Moderate Risk 1,664 39.7% 10.5% 

High Risk 312 59.6% 15.1% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 13.1% 1.6% 

Moderate Risk 1,790 38.7% 10.3% 

High Risk 329 60.2% 15.5% 
4. Re-Developed Risk Assessment 

Low Risk 776 13.4% 1.5% 

Moderate Risk 1,608 36.4% 92% 

High Risk 527 57.9% 16.9% 








The following is a summary of how the risk assessment versions compared: 


« The original risk assessment did not distinguish well between moderate and high risk 


youth overall. 


In particular, there was little difference in recidivism rates between 


moderate and high risk females and between moderate and high risk non-white youth. 


= For the total sample, the original risk assessment with re-weighted items and the re- 
developed assessment resulted in greater differentiation between low and moderate risk 
youth, and moderate and high risk youth than did the other risk assessment versions (see 


Table E3). 
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Table E3 


Percentage Increase in Rates between Risk Levels 
by Risk Assessment Version 


















































Subsequent Referral Outcome Subsequent Adjudication Outcome 
From From From 
Risk Assessment Version Low to Moderate | From Low| From Low | Moderate | From Low 
Moderate To High to High to Moderate to High to High 
1. Original Risk Assessment 105% 29% 164% 163% 29% 240% 
2s Original Risk Assessment with 153% 50% 280% 250% 44%, 403% 
Revised Cut Points 
3. Original Risk Assessment with a ‘ is 6 - . 
Re-Weighted Items 195% 56% 356% 544% 50% 869% 
4. Re-Developed Risk Assessment 172% 59% 332% 513% 84% 1,027% 











For example, the percentage increase from low to moderate is (low rate — moderate rate)/low rate. 


Note: The data shown is percentage increase, calculated by dividing the difference in rates by the rate of the lower risk level. 


« With regard to sample subgroups, the areas of concern were the amount of separation 
between moderate and high risk females, moderate and high risk youth of non-white 
ethnicities, and moderate and high risk urban youth. The re-developed risk assessment 
and the original risk assessment with re-weighted items had greater differentiation 
between moderate and high risk youth in these subgroups than did the other risk 

The re-developed risk assessment provided slightly greater 

differentiation than did the original risk assessment with re-weighted items. 


assessment versions. 


In conclusion, the original risk assessment with re-weighted items and the re-developed 


risk assessment appear to have attained the best separation between risk levels. 


While both 


assessments achieved this regardless of the youth’s gender, ethnicity, or area of residence, the re- 


developed assessment achieved slightly greater separation between moderate and high risk youth 


for females and youth of non-white ethnicity. 


The decision about modifying the original risk assessment is, however, based both on 


research and policy. A number of policy issues affect risk assessment modifications: 


= Changes to the risk assessment would need to be made in BANNER (the statewide 
information system), which could be expensive and/or delay the implementation of 


BANNER; 
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= Changes to the risk assessment may also need to be made to J-TRAC, unless those 
counties would use the original risk assessment until BANNER implementation; and 


= Changes would need to be made to the manual (which may include changes to 
definitions), and those changes communicated to officers and other staff (through training 
or other means). 
All of the altered risk assessment versions reviewed in this report are similar to the original risk 
assessment (see Appendix A); which may ease the burden of modifications, particularly those 
related to manual changes and communication of changes to staff. 

The risk assessment committee decided by majority vote to adopt the original risk 
assessment with re-weighted items and revised cut points. Many committee members expressed 
that improvements in the classification ability produced by the risk assessment versions with re- 
weighted items (versions 3 and 4) warranted changing the original assessment. In addition, some 
members preferred to weigh the risk items according to their relationship with outcomes rather 
than relying upon the existing weights, which were determined by consensus. Many indicated, 
however, that the difference in performance between the risk assessment with re-weighted items 
and the redeveloped risk assessment was not substantial enough to justify eliminating the parent 
criminality item and collapsing some risk item options. The version of the risk assessment 
adopted by the committee does not collapse categories or eliminate any items, thus no changes to 


definitions are necessary. 
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I. INTRODUCTION 

In 1998, the Missouri Office of the State Courts Administrator (OSCA) developed a 
comprehensive juvenile offender classification system that included a risk assessment, a 
classification matrix based upon risk and most serious offense, and a needs assessment. The 
classification system was implemented in 11 counties and the city of St. Louis in March of 
1999.° In July of 1999, a software product called J-TRAC, developed by the Office of State 
Courts Administrator to automate the classification system, was released to all of the 
participating implementation circuits. In December 2000, OSCA contracted with the National 
Council on Crime and Delinquency (NCCD) to conduct a second validation study of the risk 


assessment scale. 


II. BACKGROUND 
In 1995, the Missouri General Assembly passed the Juvenile Crime and Crime 
Prevention Bill. The bill, which sought to establish a more comprehensive approach to juvenile 
justice in State of Missouri, included legislative mandates for OSCA to: 
= Develop standardized assessment procedures for identifying juvenile offenders 
(subsection 5, section 1 of section 211.326 RSMo. and subsections 4 and 5 of section 
211.141 RSMo. Supp. 1995), with assessment forms developed considering racial 


disparities in the juvenile justice system (section 2 of section 211.326 RSMo.). 


= Develop a process to evaluate services and collect relevant outcome measure data 
(subsection 4, section | of section 211.326 RSMo. Supp 1995). 


= Biennially review a sample of assessment and dispositions to recommend assessment 
and disposition equity throughout the state, including any evidence of racial disparity 
in certification (subsection 5 of section 211.141 RSMo. Supp 1995). 





> Circuits 10, 11, 12, 19, 20, and 22 implemented the classification system. 
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Pursuant to the legislation, the OSCA developed, and is implementing statewide, the Missouri 
Juvenile Offender Classification System. The classification system uses a standardized risk 
assessment to classify youth according to their relative risk of re-offending, and a needs 
assessment to identify appropriate service interventions. The system also employs a 
classification matrix that guides case-management decision-making by linking offenders with 
different offense types and risk levels with a set of graduated sanctions. 

The risk assessment was developed in 1998 using a consensus approach. A Risk 
Assessment Committee (RAC) comprised of juvenile justice professionals selected the risk items 
and their associated weights based on what they believed would best classify Missouri youth 
according to offense severity and the likelihood of reoffending. OSCA and the RAC contracted 
with Dr. Kimberly Leonard, a criminologist from the University of Missouri-St. Louis, to 
empirically validate the risk assessment by determining whether, or to what extent, there was a 
relationship between total risk score and recidivism. Recidivism was defined as any new 
delinquency referral following the risk assessment. Dr. Leonard’s validation study also 
established cut-off scores to classify youth into low, moderate, or high risk categories. The 
complete classification system was subsequently implemented in the urban, rural, and suburban 
counties previously identified. 

The primary purpose of risk assessment is to classify youth according to the relative 
likelihood that they will re-offend in the future. Youth assessed as high risk should, for instance, 
have a significantly higher rate of recidivism than their low risk counterparts. By identifying 
high risk youth, agencies can protect public safety while optimizing resource allocation by 
focusing on those youth that are most likely to re-offend. A risk assessment is helpful, however, 
only if it accurately classifies youth according to their likelihood of recidivism. To ensure that 


Missouri’s risk assessment is classifying youth accurately, OSCA contracted with the National 


2 
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Council on Crime and Delinquency (NCCD) to re-validate their existing risk assessment. This 
study reflects Missouri’s ongoing commitment to implement an effective classification system 
and is consistent with the legislative requirement to periodically evaluate the effectiveness of the 
system. 
In order to determine the classification accuracy of the Missouri Juvenile Offender Risk 
Assessment, NCCD: 
1. Examined how well the original risk assessment classifies youth into low, moderate, 
and high risk groups by observing recidivism; and 
2. Analyzed the available data to determine if alterations to the original risk assessment 
could improve its performance. 
An advantage of revalidating Missouri’s existing risk assessment after its field 
implementation is that juvenile officers scored risk factors such as substance abuse and parenting 


style under actual field conditions, which reduced the likelihood of observation bias. 


Il. RESEARCH METHODOLOGY 
The research sample consisted of 2,911 youth for whom a risk assessment was completed 
between March and December 1999.° The following information was extracted for each case 


from the JTRAC database: 


1. Demographic information including ethnicity, gender, and reporting circuit; 


2: Data from the original risk assessment recorded by officers at the time of the 
sampled referral; and 


2: Outcome data about subsequent referrals and adjudications. 


° Risk assessments are completed for youth with referrals that meet the definition of legal sufficiency. 


3 
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Outcome measures for each youth were tracked for a standardized 12-month period 
following the sampled referral and included any referrals, accepted referrals, petitioned referrals, 


violent offense referrals, and adjudications (petitions found true). 


A. Sample Characteristics 

Demographic characteristics of the youth at the time of their sampled referral are shown 
in Table 1. Two thirds of the youth were white (66.5%) and male (69.3%). The majority 
(55.9%) of youth in the sample were from an urban area,’ with another 34.5% from rural 
counties (the remainder did not have a county or circuit indicated). 

Table 1 also reviews the nature of the sampled referral. The most serious allegation for 
most referrals was a misdemeanor or class C&D felony (70.7%). Only 4.1% of the sampled 


referrals were for an A or B class felony. 





7 St. Louis and St. Charles (circuits 11 and 22) are classified as urban, while the remainder of the sampled circuits is classified as 
rural (circuits 10, 12, 19 and 20). Percentages are based upon the total sample (N = 2,911) rather than the sample with county 
indicated (i.e., cases with county missing are included in the denominator). 


4 
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Table 1 


Characteristics of Sampled Referral and Youth 




















Risk Assessment Item N % 
Total Sample 2,911 100.0% 
Gender 
Male 69.3% 
Female 30.7% 
Race 
White 66.5% 
African American 915 31.4% 
Hispanic 0.3% 
American Indian 0.2% 
Asian or Pacific Islander 0.2% 
Other 1.1% 
Missing -_— = — 3 0.3% 
Urban 
Rural 34.5% 
Urban 55.9% 
Missing 9.6% 
Offense Severity 
Status and Municipal 25.2% 
Misdemeanor, C&D Felony 70.7% 
A&B Felony 4.1% 
Missing 0.1% 
Major Offense Type 
Status offense 22.3% 
Court order violation 0.3% 
Municipal charges 0.8% 
Major law offenses 74.3% 
Motor vehicle violence 0.5% 
Hazardous driving 0.5% 
Missing 34 1.2% 
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Table 2 shows the prevalence of the risk factors for the sampled youth, as indicated by 
the officer who completed the risk assessment at the time of the sampled referral. One-third 
(33.0%) of the youth were age 12 or under at the time of their first referral for a delinquency 
offense, and 50.3% had a prior referral. Less than one-fourth (21.3%) had a prior out-of-home 
placement, 16.0% were maltreated as a child, and 28.1% had a current substance use problem. 


Slightly more than half (51.0%) had school-related behavior problems. 

















Table 2 
Sample Distribution of Risk Assessment Items 

Risk Assessment Item N % 
Total Sample 2,911 100.0% 
Age at First Referral 

16 12.5% 

13,14 or 15 54.5% 

12 and under 33.0% 
Prior Referrals 


None 1,446 49.7% 
One or more 1,465 50.3% 


Assault Referrals (Prior or Present) 





No prior or present assault referral 68.4% 
One or more misdemeanor assault 24.2% 
One or more felony assault 7.5% 


History of Placement 
No prior out-of-home 2,290 78.7% 
Prior out-of-home 21.3% 


Peer Relationships 





Neutral influence 50.3% 
Negative influence 42.6% 
Strong negative influence 7.1% 





History of Child Abuse/ Neglect 
No prior CA/N 2,444 84.0% 
Prior CA/N history 16.0% 








Substance Abuse 
No problem 71.9% 
Moderate problem 25.6% 
Severe dependence 2.5% 


School Behavior Problems 
No or minor problems 





Moderate problems 





Parental Management Style 
Positive management 57.6% 
Moderately ineffective management 984 33.8% 
Severely ineffective management 249 8.6% 
Parents’ Criminal History 


No prior incarceration 2,260 77.6% 


Prior incarceration 651 22.4% 


6 
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Table 3 shows the rates of recidivism for the overall sample and subgroups during the 
standardized 12-month follow-up period (i.e., base rates). For the entire sample, 34.1% of the 
youth had received a new referral of any type, 26.1% had an accepted referral of some type, 
11.1% had a petitioned referral, 9.8% had a referral for a violent offense, and 8.6% had an 


adjudication (petition found true) subsequent to the sampled referral. 




































































Table 3 
Risk Re-Validation Sample Base Rates 

Sample Accepted Petitioned Violent Offense 

Group N Any Referral Referral Referral Referral Adjudication 
Sample 2,911 34.1% 26.1% 11.1% 9.8% 8.6% 
White 1,935 27.7% 23.6% 7.9% 7.7% 7.3% 
Non- White 968 47.1% 31.1% 17.4% 13.9% 10.8% 
Rural 1,005 34.3% 30.3% 94% 9.5% 9.0% 
Urban 1,626 33.8% 24.0% 12.6% 9.9% 8.8% 
Male 2,018 38.2% 29.5% 13.3% 11.0% 10.2% 
Female 893 25.1% 18.4% 6.2% 7.1% 4.8% 














Note: All differences are significant with 95% confidence with two exceptions; the differences in re-referral (any 
referrals) and subsequent adjudications rates for urban vs. rural circuits is not significant. 


Recidivism rates differ significantly, however, between groups defined by ethnicity and 
gender. For example, 27.7% of white youth had a delinquency referral in the follow-up period, 
while 47.1% of non-white youth had a subsequent referral. The difference between genders is 
less pronounced; 38.2% of males had a subsequent referral while 25.1% of females were 
referred. 

It is easier to construct a risk assessment that classifies subgroups similarly when the base 
rates of the outcome measure are similar across groups. In comparing recidivism rates by 


ethnicity, the outcome of subsequent adjudications shows less difference between groups than 


other outcomes (a 3.5% difference between whites and non-whites). Males, however, had more 


7 
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than twice the rate of subsequent adjudication than females (a 112% percentage increase).* The 
outcome of subsequent referral shows less disparity by gender; the re-referral rate for males is 
52% greater than that of females (38.2% and 25.1% respectively). 

Table 4 shows recidivism rates across the five outcome measures by circuit type and 
ethnicity. The proportions by ethnicity were not the same within geographic locations; urban 
areas had a greater proportion of non-white youth than did rural areas.’ Given this, it is not 
unexpected that most recidivism rate differences between ethnic groups within rural circuits 
were not significant, and differences between ethnic groups within urban circuits were 
significant. Adjudication is the measure of recidivism most similar across circuit type and 
ethnicity. White youth in rural circuits had an adjudication rate of 9.1%, while non-white rural 
youth had a lower rate (5.6%). White urban youth had a true petition rate of 6.0%, and non- 


white urban youth had a rate of 12.4%, the highest of the four groups. 






































Table 4 
Risk Re-Validation Sample Base Rates 
Savane Subsea Any Accepted Petitioned Violent Offense 
eadestau cea N Referral Referral Referral Referral Adjudication 

cea White 909 33.7% 29.9% 9.6% 9.4% 9.1% 
i Non-White 90 41.4% 34.4% 5.6% 11.1% 5.6% 
Urban White 906 22.0% 17.8% 6.7% 6.3% 6.0% 
Non-White 720 48.6% 31.8% 20.0% 14.4% 12.4% 
































Note: Differences between ethnic groups in the urban circuits are significant with 95% confidence. Differences between 
ethnic groups within rural circuits are not significant, with the exception of subsequent referrals (any referrals), which is 
significant at the .10 level. Total N size is 2,368 (missing data is 18.6% or 543 cases). 


While the results from all five of the outcome measures were reviewed during the 


analyses, subsequent referral (of any type) and adjudications were the primary outcomes 


8 This comparison (a percentage increase) is calculated by dividing the difference in rates by the lower rate. For example, the 
(high risk rate — low risk rate) is divided by the low risk rate. The purpose of this is to compare differences while controlling for 
the lower rate. For example, the difference between low-moderate risk and moderate-high risk might both be 10%, but the 
relative increase would be very different. 

° Of youth from an urban area, 55.7% were white while 44.3% were of another ethnicity. In contrast, 91% of youth from rural 
areas were white. 
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measures analyzed. Subsequent referral was selected as a primary outcome measure because the 
previous validation study conducted by Dr. Leonard used this measure, and retaining it provides 
a method of comparison. Subsequent adjudication was considered an important measure because 
court involvement indicates substantial evidence that the youth committed the alleged offense, 


and adjudication rates are less disparate across key subgroups. 


B. Method of Analysis 

The analysis presented in this report had two goals: 1) to assess the performance of the 
original risk assessment, and 2) to determine if changes to the extensive risk assessment would 
improve its classification accuracy. To determine how effectively the existing risk assessment 
worked, NCCD first examined the relationship between the existing risk classification and 
subsequent delinquency referrals. This analysis used cross tabulations between risk 
classifications and the five outcome measures previously mentioned for both the overall sample 
and for key subgroups based on gender, ethnicity, and type of residence (rural versus urban). 

The second step in the analysis was to determine whether or not alterations made to the 
existing assessment could improve performance. This involved an extensive evaluation of risk 
items and their associated weights (the number of points received when an item was found true) 
relative to recidivism outcomes, and an evaluation of the efficacy of the cut points that classify 
youth as low, moderate or high risk. 

The third step toward achieving the goals of the research involved constructing a 
revalidated actuarial risk assessment instrument. The revalidated instrument was developed by 
observing the actuarial relationship between youth and family characteristics observed at the 


time of the sample referral and subsequent delinquency referrals received for the youth overall 
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and for subgroups previously defined. This analysis involved both bivariate and multi-variate 


statistical techniques. ° 


IV. FINDINGS 

An effective and valid risk assessment has progressively higher recidivism rates that 
correspond to each increase in risk classification level across multiple outcomes. Ideally, the 
rates between consecutive risk levels maximize the separation between the high and low risk 
groups, as well as between consecutive risk groups. In other words, each increase in risk level 
should correspond to an increase in recidivism, across outcomes, that is significantly greater. 
The best way to assess the performance of the risk assessment versions, then, is to compare the 
separation between risk levels: 

1. between the low and high risk groups; and 

2. between consecutive groups. 

This comparison was made for the total sample as well as subgroups defined by youths’ 
ethnicity, gender, and geographic location. 

The risk assessment currently in use by officers in participating circuits provided risk 
classifications that identify youth with significantly different rates of subsequent delinquency for 
most outcomes. However, analysis of the risk items and the cut points that define the risk 
classifications suggested that certain modifications could improve the classification accuracy of 


the assessment. In addition, a full-scale re-examination of the relationships between the risk 


'0 A variety of statistical methods could be used to conduct the analyses described. A prior study by Simon (1971) and an 
exhaustive study by Gottfredson and Gottfredson (1979), later confirmed by other researchers (see Wilbanks, 1985 and Benda, 
1987), found that less precise methods of statistical evaluation (including bivariate analyses or least squares regression) often 
produce the best overall result. These procedures were employed in this analysis. 
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items officers observed at the time of the youth’s referral and subsequent delinquency resulted in 
an alternative risk assessment that differed from the current one. 

The following review of the findings compares the distribution and performance of the 
original risk assessment to alternative versions of the assessment derived from this analysis of 


the overall sample and for sample subgroups. The risk assessment versions presented are: 


1. The original risk assessment: the risk assessment currently used by Missouri 
officers. 


2. The original risk assessment with revised cut points: the original risk assessment 
with no changes to the items but the revised cut points are defined as low risk, 0-4; 
moderate risk, 5-16; and high risk, 17 points or more. 


3. The original risk assessment with re-weighted items: the original risk assessment 
with item weights reduced for all risk items, and cut points altered relative to the new 
distribution of risk scores. To clarify, this version retains all risk items and choices 
per item on the original risk assessment, but changes the number of points assigned 
when an item is found true. These changes are reviewed in Appendix A. 


4. The re-developed risk assessment: the risk assessment that resulted from a complete 
multi-variate analysis of the data. This assessment includes many of the re-weighted 
items that are present in the previous version of the assessment, but includes 
additional changes. These additional changes are: a) replacing the current assault 
referral item to a yes/no item (dropping the separation between felony and 
misdemeanor assault referrals); b) collapsing moderate and severe school problems 
into one category for the school behavior item; c) similarly collapsing the parental 
management item; and d) eliminating the parents’ criminality item. This assessment 
is also shown in Appendix A. 


A. Risk Assessment Classification Findings for the Overall Sample 

These versions of the risk assessment, as expected, resulted in slightly different 
distributions by risk level (see Table 5). When applied to all sampled youth, the original risk 
assessment classified 49.1% of the youth as low risk, 30.5% as moderate risk and 20.4% as high 
risk. In comparison, the other risk assessment versions classified a higher proportion of the 


sampled youth as moderate risk. The original risk assessment with revised cut points classified 
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57.2% of youth as moderate risk and 10.7% as high risk. The original risk assessment with 
revised cut points and re-weighted items similarly classified 62.8% of youth as moderate risk and 
11.3% as high risk. The re-developed assessment classified 55.2% of the youth as moderate risk 


and 18.1% as high risk. 




















Table 5 
Risk Level Distribution 
by Risk Assessment Version 
Low Risk Moderate Risk High Risk Total 
N % N % N % N % 
1. Original Risk Assessment 1,429 | 49.1% 888 30.5% 594 20.4% 2,911 100.0% 





2. Original Risk Assessment with 


0 0 0 0 
Revised Cat Paints 935 | 32.1% 1,664 | 57.2% 312 10.7% | 2,911 | 100.0% 





3. Original Risk Assessment with 


Re-Weighted Items!! 753 | 25.9% 1,829 | 62.8% 329 11.3% | 2,911 | 100.0% 











4. Re-Developed Risk Assessment | 776 26.7% 1,608 55.2% 527 18.1% | 2,911 100.0% 



































The following tables review the five outcomes by the version of the risk assessment for 
the overall sample. Table 6 and Figure 1 show that when the original risk assessment is applied, 
youth classified as low risk had a re-referral rate of 20.6%, youth classified as moderate risk had 
a re-referral rate of 42.2%, and 54.5% of youth classified as high risk had a subsequent referral. 
The re-referral rate for high risk youth was 1.6 times greater than the rate for low risk youth.’ 

The differences in re-referral rates among youth classified at each risk level were greater 
for the altered versions of the risk assessment. When the original risk assessment with revised 
cut points is applied, youth classified as low risk had a 15.7% re-referral rate, while 59.6% of 
high risk youth had a subsequent referral (2.8 times greater than that of low risk youth). When 


applying the original risk assessment with revised cut points and re-weighted items, 13.1% of 


' This version also has, by necessity, revised cut points. For ease of reference, however, future tables will use the label ‘the 
Original Risk Assessment with Re-Weighted Items.’ 


'? This comparison (a percentage increase) is calculated by dividing the difference in rates by the lower rate. For example, the 
(high risk rate — low risk rate) is divided by the low risk rate. The purpose of this comparison is to enable comparisons of 
differences, while controlling for the lower rate. For example, the difference between low-moderate risk and moderate-high risk 
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youth classified as low risk and 60.2% of youth classified as high risk had a subsequent referral 
(3.6 times greater than that of low risk youth). Youth classified as low risk by applying the re- 
developed tool had a re-referral rate of 13.4%, while high risk youth had a re-referral rate of 


57.9% (3.3 times greater than the rate for low risk youth). 










































































Table 6 
Findings for Subsequent Referral 
by Risk Assessment Version 
Subsequent Referral 
Total N N % 

Total Sample 2,911 997 35.5% 
1. Original Risk Assessment 

Low Risk 1,429 295 20.6% 

Moderate Risk 888 375 42.2% 

High Risk 594 324 54.5% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 147 15.7% 

Moderate Risk 1,664 661 39.7% 

High Risk 312 186 59.6% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 104 13.1% 

Moderate Risk 1,790 692 38.7% 

High Risk 329 198 60.2% 
4. Re-Developed Risk Assessment 

Low Risk 776 104 13.4% 

Moderate Risk 1,608 585 36.4% 

High Risk 527 305 57.9% 





























might both be 10%, but the percentage increase would be very different. 
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Figure 1 


Findings for Subsequent Referral 
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Cut Points Weighted Items 











4 Low H Moderate @ High 








Table 7 and Figure 2 similarly compare classification findings when the outcome is 
referrals accepted for investigation. Using the original risk assessment, 16.9% of low risk youth 
had a subsequent accepted referral in the follow-up period compared to 39.7% of high risk youth. 
For the original risk assessment with revised cut points, corresponding outcome rates were 
13.2% and 41.0%. Both the original risk assessment with re-weighted items and the re- 
developed risk assessment showed a greater difference in re-referral rates between low and high 
risk youth when compared to those the other two assessment versions (10.4% and 40.7% for the 
original risk assessment with re-weighted items; 10.4% and 40.0% for the re-developed 


assessment). 
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Table 7 
Findings for Subsequent Accepted Referral 
by Risk Assessment Version 
Subsequent Accepted Referral 
Total N N % 

Total Sample 2,911 759 26.1% 
1. Original Risk Assessment 

Low Risk 1,429 241 16.9% 

Moderate Risk 888 282 31.8% 

High Risk 594 236 39.7% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 123 13.2% 

Moderate Risk 1,664 508 30.5% 

High Risk 312 128 41.0% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 82 10.4% 

Moderate Risk 1,790 543 30.3% 

High Risk 329 134 40.7% 
4. Re-Developed Risk Assessment 

Low Risk 776 81 10.4% 

Moderate Risk 1,608 467 29.0% 

High Risk 527 211 40.0% 

Figure 2 


Findings for Subsequent Accepted Referral 
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The re-developed risk assessment and the original risk assessment with re-weighted items 
also achieved greater separation between youth classified as low risk and high risk when the 
observed outcome is subsequent petitioned referrals. Table 8 and Figure 3 show that when the 
original risk assessment was applied, 4.8% of youth classified as low risk had a petitioned 
referral, while 21.9% of high risk youth had a referral petitioned (3.6 times greater than the rate 
for low risk youth). With revised cut points, low risk youth had a 3.5% recidivism rate versus 
24.0% for high risk youth (5.8 times greater than the rate for low risk youth). Using the re- 
weighted or re-developed risk assessment, the rate of subsequent petitioned referrals for high risk 


youth was nearly 12 times greater than that of low risk youth. 


































































































Table 8 
Findings for Subsequent Petitioned Referral 
by Risk Assessment Version 
Subsequent Petitioned Referral 
Total N N % 

Total Sample 2,911 323 11.1% 
1. Original Risk Assessment 

Low Risk 1,429 69 4.8% 

Moderate Risk 888 124 14.0% 

High Risk 594 130 21.9% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 33 3.5% 

Moderate Risk 1,664 215 12.9% 

High Risk 312 75 24.0% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 16 2.0% 

Moderate Risk 1,790 225 12.6% 

High Risk 329 82 24.9% 
4. Re-Developed Risk Assessment 

Low Risk 776 15 1.9% 

Moderate Risk 1,608 181 11.3% 

High Risk 527 127 24.1% 
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Figure 3 


Findings for Subsequent Petitioned Referral 
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When the outcome measure is violent offense referrals, the results were similar to those 
found with the previous outcome measures (see Table 9 and Figure 4). Applying the original 
risk assessment, 16.3% of youth classified as high risk and 5.3% of low risk youth had a 
subsequent violent offense referral. With revised cut points, 3.9% of low risk youth and 19.6% 
of high risk youth had a subsequent referral for a violent offense. When the original risk 
assessment with re-weighted items and the re-developed risk assessment was applied, high risk 
youth had a subsequent violent offense referral rate that was more than five times greater than 
that of low risk youth (5.5 times greater for the original risk assessment with re-weighted items 


and 5.2 greater for the re-developed assessment). 
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Table 9 
Findings for Subsequent Violent Offense Referral 
by Risk Assessment Version 
Subsequent Violent Offense Referral 
Total N N % 

Total Sample 2,911 284 9.8% 
1. Original Risk Assessment 

Low Risk 1,429 76 5.3% 

Moderate Risk 888 111 12.5% 

High Risk 594 97 16.3% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 36 3.9% 

Moderate Risk 1,664 187 11.2% 

High Risk 312 61 19.6% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 22 2.8% 

Moderate Risk 1,790 202 11.3% 

High Risk 329 60 18.2% 
4. Re-Developed Risk Assessment 

Low Risk 776 21 2.7% 

Moderate Risk 1,608 174 10.8% 

High Risk 527 89 16.9% 

Figure 4 
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Table 10 and Figure 5 illustrate the results of the risk assessment versions when the 
outcome measure is subsequent adjudication. After applying the original risk assessment, 4.3% 
of youth classified as low risk had a referral that lead to adjudication, while 11.3% of youth 
classified as moderate risk and 14.6% of youth classified as high risk had a subsequent 
adjudication. For the original risk assessment with revised cut points, the differences in 
subsequent adjudication rates between consecutive risk levels were somewhat improved with 
3.0% of youth classified as low risk, 10.5% of moderate risk youth and 15.1% of high risk youth 
having been adjudicated. Using the original risk assessment with re-weighted items yielded 
further improvement; 1.6% of low risk, 10.3% of moderate risk, and 15.5% of high risk youth 
had a subsequent referral that resulted in adjudication. When classified by the re-developed risk 
assessment, 1.5% of low risk youth, 9.2% of moderate risk youth, and 16.9% of high risk youth 


were subsequently adjudicated. 


































































































Table 10 
Findings for Subsequent Adjudication 
by Risk Assessment Version 
Subsequent Adjudication 
Total N N % 

Total Sample 2,911 249 8.6% 
1. Original Risk Assessment 

Low Risk 1,429 62 4.3% 

Moderate Risk 888 100 11.3% 

High Risk 594 87 14.6% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 28 3.0% 

Moderate Risk 1,664 174 10.5% 

High Risk 312 47 15.1% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 13 1.6% 

Moderate Risk 1,790 185 10.3% 

High Risk 329 51 15.5% 
4. Re-Developed Risk Assessment 

Low Risk 776 12 1.5% 

Moderate Risk 1,608 148 9.2% 

High Risk 527 89 16.9% 
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Figure 5 


Findings for Subsequent Adjudication 
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The following sections review the results of the risk assessment classifications by sample 


subgroups. 
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B. Risk Assessment Classification Findings by Gender 

Tables 11 through 13 review findings for the four risk assessment versions for males and 
females. Table 11 shows that the risk level distributions for males and females were similar 
across all risk scale versions, with proportionately more females classified as low risk and more 


males classified as moderate and high risk. 






















































































Table 11 
Risk Level Distribution 
by Youth Gender 
Male Female Overall 
N % N % N % 

Total Sample 2,018 | 100.0% 893 100.0% | 2,911 100.0% 
1. Original Risk Assessment 

Low Risk 927 45.9% 502 56.2% 1,429 49.1% 

Moderate Risk 659 32.7% 229 25.6% 888 30.5% 

High Risk 432 21.4% 162 18.1% 594 20.4% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 592 29.3% 343 38.4% 935 32.1% 

Moderate Risk 1,190 59.0% 474 43.1% 1,664 57.2% 

High Risk 236 11.7% 76 8.5% 312 10.7% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 497 24.6% 295 33.0% 792 27.2% 

Moderate Risk 1,272 63.0% 518 58.0% 1,790 61.5% 

High Risk 279 12.3% 80 9.0% 329 11.3% 
4. Re-Developed Risk Assessment 

Low Risk 484 24.0% 292 32.7% 776 26.7% 

Moderate Risk 1,140 56.5% 468 52.4% 1,608 55.2% 

High Risk 394 19.5% 133 14.9% 527 18.1% 























Table 12 presents findings for the five risk assessment versions by gender when the 
outcome is subsequent referral (any type). Overall, males were 41% more likely to have a 
subsequent referral than were females (base rates are 38.2% and 27.1% respectively). Despite 
the significant difference in base rates, all four risk assessment versions classified both male and 
female youth such that an increase in the risk level corresponded to an increase in the re-referral 
rate. 

The original risk assessment with re-weighted items and the re-developed risk assessment 


versions, however, more accurately classified youth across genders than did the other versions. 
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Under the original risk assessment, the re-referral rate for high risk females was 43.8% and 
59.6% for high risk males. In fact, the re-referral rate for high risk females was closer to that of 
moderate risk males (44.8%). 

In comparison, when the original risk assessment with re-weighted items was applied, 
low risk males had a re-referral rate of 15.3%, while moderate risk males had a rate of 41.9% and 
high risk males a rate of 64.7%. Under this version, low risk females had a re-referral rate of 
9.5%, moderate risk females a rate of 30.7% and high-risk females a re-referral rate of 46.3%. 
The original assessment with re-weighted items produced the greatest distinction between low 
and moderate risk males and females. 

The re-developed assessment classified males such that low risk males had a re-referral 
rate of 15.7% and high risk males had a re-referral rate of 60.7%. Females classified as low risk 
had a re-referral rate of 9.6%, while high risk females had a re-referral rate of 49.6%. The re- 
developed risk assessment had the most similar high risk re-referral rates across gender, and the 


greatest distinction between the rates for moderate risk males and high risk females. 














































































































Table 12 
Findings for Subsequent Referral by Youth Gender 
Male Female 
Total Subsequent Referral Total Subsequent Referral 
N N % N N % 
Total Sample 2,018 770 38.2% 893 224 27.1% 
1. Original Risk Assessment 
Low Risk 927 222 23.9% 502 73 14.5% 
Moderate Risk 659 295 44.8% 229 80 34.9% 
High Risk 432 353 59.6% 162 71 43.8% 
2. Original Risk Assessment with Revised Cut Points 
Low Risk 592 109 18.4% 343 38 11.1% 
Moderate Risk 1,190 511 42.9% 474 150 31.6% 
High Risk 236 150 63.6% 76 36 47.4% 
3. Original Risk Assessment with Re-Weighted Items 
Low Risk 497 76 15.3% 295 28 9.5% 
Moderate Risk 1,272 533 41.9% 518 159 30.7% 
High Risk 249 161 64.7% 80 37 46.3% 
4. Re-Developed Risk Assessment 
Low Risk 484 76 15.7% 292 28 9.6% 
Moderate Risk 1,140 455 39.9% 468 130 27.8% 
High Risk 394 239 60.7% 133 66 49.6% 
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Table 13 makes a similar comparison for subsequent adjudication. Again, achieving 
similar rates within risk levels for both males and females was made difficult by the difference in 
base rates (males were twice as likely than females to have a subsequent adjudication). 

All of the risk assessment versions classified males and females such that an increase in 
risk level had a corresponding increase in the rate of subsequent adjudication. Only the re- 
developed assessment, however, was able to attain a high degree of separation between moderate 
and high risk females; that is, high risk females have double the rate of subsequent adjudications 


of moderate risk females. The increase from moderate to high risk was less than 50% for other 

















































































































versions. 
Table 13 
Findings for Subsequent Adjudication 
by Youth Gender 
Male Female 
Subsequent Subsequent 
Total Adjudication Total Adjudication 
N N % N N % 

Total Sample 2,018 206 10.2% 893 3 4.8% 
1. Original Risk Assessment 

Low Risk 927 52 5.6% 502 10 2.0% 

Moderate Risk 659 82 12.4% 229 18 7.9% 

High Risk 432 72 16.7% 162 15 9.3% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 592 21 3.5% 343 7 2.0% 

Moderate Risk 1,190 145 12.2% 474 29 6.1% 

High Risk 236 40 16.9% 76 7 9.2% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 497 10 2.0% 295 3 1.0% 

Moderate Risk 1,272 152 11.9% 518 33 6.4% 

High Risk 249 44 17.7% 80 7 8.8% 
4. Re-Developed Risk Assessment 

Low Risk 484 11 2.3% 292 1 0.3% 

Moderate Risk 1,140 122 10.7% 468 26 5.6% 

High Risk 394 73 18.5% 133 16 12.0% 
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C. Risk Assessment Classification Findings by Ethnicity of the Youth 


Tables 14 through 16 present recidivism findings for the risk assessment version for 


white youth and all other ethnic groups (i.e., non-white). Each version of the risk assessment 


classified a higher proportion of non-white youth as high risk (see Table 14). The original risk 


assessment classified 2.4 times more non-white youth as high risk compared to the proportion of 


white youth classified high risk. The original risk assessment with revised cut points and with 


re-weighted items, however, classified three times as many non-white youth as high risk. The re- 


developed risk assessment, like the original risk assessment, classified twice as many non-white 




































































youth as high risk. 
Table 14 
Risk Level Distribution 
by Youth Ethnicity 
White Non-White Overall 
N % N % N % 

Total Sample 1,935 100.0% 968 100.0% | 2,911 100.0% 
1. Original Risk Assessment 

Low Risk 1,124 58.1% 299 30.9% 1,429 49.1% 

Moderate Risk 543 28.1% 344 35.5% 888 30.5% 

High Risk 268 13.9% 325 33.6% 594 20.4% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 777 40.2% 152 15.7% 935 32.1% 

Moderate Risk 1,038 53.6% 625 64.6% 1,664 57.2% 

High Risk 120 6.2% 191 19.7% 312 10.7% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 677 35.0% 111 11.5% 792 27.2% 

Moderate Risk 1,132 58.5% 655 67.7% 1,790 61.5% 

High Risk 126 6.5% 202 20.9% 329 11.3% 
4. Re-Developed Risk Assessment 

Low Risk 654 33.8% 118 12.2% 776 26.7% 

Moderate Risk 1,047 54.1% 559 57.7% 1,608 55.2% 

High Risk 234 12.1% 291 30.1% 527 18.1% 



































Note: Eight cases lacked information regarding the youth’s race. 
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Table 15 shows that when comparing risk assessment performance by ethnicity for 
subsequent referrals, the original risk assessment did not differentiate between moderate and 
high risk non-white youth as well as the other risk assessment versions did. When the original 
risk assessment was applied, the increase from moderate to high risk for non-white youth 
corresponded to a 25% increase in the re-referral rate (from 49.1% to 61.5%). Among non-white 
youth, the percentage increase was 43% for the original risk assessment with revised cut points 
(46.4% to 66.5%), 49% for the original risk assessment with re-weighted items (45.2% to 
67.3%), and 52% for the re-developed risk assessment (42.9% to 65.3%). The re-developed risk 
assessment and the original assessment with re-weighted items provided better distinction 
between low and moderate risk non-white and white youth, with the latter providing the best 


distinction between the risk groups by ethnicity. 

















































































































Table 15 
Findings for Subsequent Referral 
by Youth Ethnicity 
White Non-White 
Total Subsequent Referral Total Subsequent Referral 
N N % N N % 

Total Sample 1,935 536 27.7% 968 456 47.1% 
1. Original Risk Assessment 

Low Risk 1,124 208 18.5% 299 7 29.1% 

Moderate Risk 543 205 37.8% 344 169 49.1% 

High Risk 268 123 45.9% 325 200 61.5% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 777 108 13.9% 152 39 25.7% 

Moderate Risk 1,038 370 35.6% 625 290 46.4% 

High Risk 120 58 48.3% 191 127 66.5% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 677 80 11.8% 111 24 21.6% 

Moderate Risk 1,132 395 34.9% 655 296 45.2% 

High Risk 126 61 48.4% 202 136 67.3% 
4. Re-Developed Risk Assessment 

Low Risk 654 78 11.9% 118 26 22.0% 

Moderate Risk 1,047 345 33.0% 559 240 42.9% 

High Risk 234 113 48.3% 291 190 65.3% 
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A similar comparison was made for the outcome of subsequent adjudication in Table 16. 
As with the previous comparison, the re-developed risk assessment and the original risk 
assessment with re-weighted items achieved better separation between risk levels than did the 
other risk assessment versions. The re-developed risk assessment had slightly greater separation 
between moderate and high risk non-white youth; the corresponding percentage increase for the 
re-developed risk assessment was 27.0%, compared to 15.5% for the risk assessment with re- 


weighted items (a difference of 3% and 1.8% respectively). 

















































































































Table 16 
Findings for Subsequent Adjudication 
by Youth Ethnicity 
White Non-White 
Subsequent Subsequent 
Total Adjudication Total Adjudication 
N N % N N % 

Total Sample 1,935 142 7.3% 968 105 10.8% 
1. Original Risk Assessment 

Low Risk 1,124 43 3.8% 299 19 6.4% 

Moderate Risk 543 56 10.3% 344 43 12.5% 

High Risk 268 43 16.0% 325 43 13.2% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 777 20 2.6% 152 8 5.3% 

Moderate Risk 1,038 102 9.8% 625 71 11.4% 

High Risk 120 20 16.7% 191 26 13.6% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 677 11 1.6% 11] 2 1.8% 

Moderate Risk 1,132 108 95% 655 76 11.6% 

High Risk 126 23 18.3% 202 27 13.4% 
4. Re-Developed Risk Assessment 

Low Risk 654 10 1.5% 118 2 1.7% 

Moderate Risk 1,047 86 8.2% 559 62 11.1% 

High Risk 234 46 19.7% 291 41 14.1% 
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D. Risk Assessment Classification Findings by Geographic Location 

Over half of the youth in the sample (55.8%) were from an urban county.'? Of youth 
from an urban area, 55.7% were white, and 44.3% were of another ethnicity. In contrast, 91% of 
youth from rural areas were white. Given this difference in the urban and rural populations, the 
findings when comparing risk assessment performance for youth residing in urban vs. rural 
counties are related to those reviewed previously for white and non-white youth. 

Table 17 shows the distribution by risk level when the risk assessment versions were 
applied to youth living in urban and rural areas. Regardless of the type of risk assessment 
version, urban youth were more likely to be classified as high risk than were youth from rural 
areas. For example, under the original risk assessment, most rural youth (55.9%) were classified 
as low risk, while a slightly lower proportion (45.7%) of urban youth were classified as low risk. 
Nearly one-fifth (17.3%) of rural youth and 21.9% of urban youth were classified as high risk. 


This pattern of distribution also appeared when the revised risk assessments were applied. 





'3 As mentioned previously, St. Louis and St. Charles (circuits 11 and 22) are classified as urban, while the remainder of the 
sampled circuits is classified as rural (circuits 10, 12, 19 and 20). Percentages are based upon the total sample (N = 2,911) rather 
than the sample with county indicated (1.e., cases with county missing are included in the denominator). 
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Table 17 
Risk Level Distribution 
by Geographic Location 
Rural Urban Overall 
N % N % N % 

Total Sample 1,005 100.0% | 1,626 | 100.0% | 2,911 100.0% 
1. Original Risk Assessment 

Low Risk 562 55.9% 743 45.7% 1,429 49.1% 

Moderate Risk 269 26.8% 527 32.4% 888 30.5% 

High Risk 174 17.3% 356 21.9% 594 20.4% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 377 37.5% 492 30.3% 935 32.1% 

Moderate Risk 540 53.7% 946 58.2% 1,664 57.2% 

High Risk 88 8.8% 188 11.6% 312 10.7% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 329 32.7% 410 25.2% 792 27.2% 

Moderate Risk 591 58.8% 1,004 61.7% 1,790 61.5% 

High Risk 85 8.5% 212 13.0% 329 11.3% 
4. Re-Developed Risk Assessment 

Low Risk 331 32.9% 394 24.2% 776 26.7% 

Moderate Risk 527 52.4% 896 55.1% 1,608 55.2% 

High Risk 147 14.6% 336 20.7% 527 18.1% 























Note: 280 cases lacked information about county of residence. 


As with the previous comparisons across sample subgroups, each risk assessment version 
classified youth such that an increase in risk corresponds to an increase in recidivism. This was 
true whether the outcome was subsequent referral or subsequent adjudication in the follow-up 
period (see Tables 18 and 19). 

Each of the risk assessment versions provided strong distinction between low and 
moderate risk cases in both the rural and the urban samples. That is, regardless of the sample 
group and the risk assessment applied, re-referral rates were nearly double with the increase to 
moderate risk (see Table 18). The increase in recidivism rates, however, was not as dramatic 
when rural and urban youth moved from moderate to high risk. For example, under the original 
risk assessment, moving from moderate to high risk corresponded to a 14.0% increase in re- 
referral rates for rural youth (from 45.4% to 51.7%) and a 37.2% increase for urban youth 


41.0% to 56.2%). The percentage increase for urban and rural youth was greater for the revised 
Pp 
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risk assessments, and greatest for the re-developed risk assessment (with a percentage increase of 


45.0% for rural youth and 67.0% for urban youth). 
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Table 18 
Findings for Subsequent Referral 
by Geographic Location 
Rural Urban 
Total Subsequent Referral Total Subsequent Referral 
N N % N N % 

Total Sample 1,005 345 34.3% 1,626 549 33.8% 
1. Original Risk Assessment 

Low Risk 562 133 23.7% 743 133 17.9% 

Moderate Risk 269 122 45.4% 527 216 41.0% 

High Risk 174 90 51.7% 356 200 56.2% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 377 67 17.8% 492 65 13.2% 

Moderate Risk 540 232 43.0% 946 363 38.4% 

High Risk 88 46 52.3% 188 121 64.4% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 329 48 46.6% 410 44 10.7% 

Moderate Risk 591 251 42.5% 1,004 373 37.2% 

High Risk 85 46 54.1% 212 132 62.3% 
4. Re-Developed Risk Assessment 

Low Risk 331 48 14.5% 394 46 11.7% 

Moderate Risk 527 211 40.0% 896 309 34.5% 

High Risk 147 86 58.5% 336 194 57.7% 
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This same pattern was evident when the outcome is subsequent adjudication in the 
follow-up period (see Table 19). Under the original risk assessment, moving from moderate to 
high risk corresponded to a 33.0% increase in rates for rural youth (from 13.4% to 17.8%) and a 
30.0% increase for urban youth (11.0% to 14.3%). The percentage increase for urban and rural 
youth was greater for the revised risk assessments, and greatest for the re-developed risk 


assessment (with a percentage increase of 156.0% for rural youth and 57.0% for urban youth). 

















































































































Table 19 
Findings for Subsequent Adjudication 
by Geographic Location 
Rural Urban 
Subsequent Subsequent 
ee Adjudication pe Adjudication 
N % N % 

Total Sample 1,005 90 9.0% 1,626 143 8.8% 
1. Original Risk Assessment 

Low Risk 562 23 4.1% 743 34 4.6% 

Moderate Risk 269 36 13.4% 527 58 11.0% 

High Risk 174 31 17.8% 356 51 14.3% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 377 11 2.9% 492 15 3.0% 

Moderate Risk 540 62 11.5% 946 102 10.8% 

High Risk 88 17 19.3% 188 26 13.8% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 329 7 2.1% 410 4 1.0% 

Moderate Risk 591 65 11.0% 1,004 109 10.9% 

High Risk 85 18 21.2% 212 30 14.2% 
4. Re-Developed Risk Assessment 

Low Risk 331 6 1.8% 394 5 1.3% 

Moderate Risk 527 49 9.3% 896 87 9.7% 

High Risk 147 35 23.8% 336 51 15.2% 
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Vv. SUMMARY 


Given that the goal of risk assessment is to classify youth according to the likelihood that 


they will re-offend in the future, each increase in risk level should correspond to a significant 


increase in recidivism, across outcomes. An effective risk assessment is one that maximizes the 


separation between recidivism rates for the high and low risk groups, as well as between rates for 


consecutive risk groups. 


The best way to assess the performance of the risk assessment versions, then, is to 


compare the separation between risk levels. Following is a summary of how the risk assessment 


versions compare: 


« The original risk assessment did not distinguish well between moderate and high risk 
youth overall. In particular, there was little difference in recidivism rates between 
moderate and high risk females and between moderate and high risk non-white youth. 


« For the total sample, the original risk assessment with re-weighted items and the re- 
developed assessment resulted in greater differentiation between low and moderate 
risk youth, and moderate and high risk youth than did the other risk assessment 
versions (see Table 20). 








Table 20 


Percentage Increase in Rates between Risk Levels 
by Risk Assessment Version 





Subsequent Referral Outcome 


Subsequent Adjudication Outcome 


























From From From 
Risk Assessment Version Low to Moderate From Low | From Low to) Moderate From Low 
Moderate to High to High Moderate to High to High 
1. Original Risk Assessment 105% 29% 164% 163% 29% 240% 
2. Original Risk Assessment ° F iy 5 & 3 
of: Revised Cut Points 153% 50% 280% 250% 44% 403% 
3. Original Risk Assessment ¥ 6 é 6 é i 
with Re-Weiphited Items 195% 56% 356% 544% 50% 69% 
See epee ne 172% 59% 332% 513% 84% 1,027% 


Assessment 
































Note: The data shown is percentage increase, calculated by dividing the difference in rates by the rate of the lower 
risk level. For example, the percentage increase from low to moderate is (low rate — moderate rate)/low rate. 
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« With regard to sample subgroups, the areas of concern were the amount of separation 
between moderate and high risk females, moderate and high risk youth of non-white 
ethnicities, and moderate and high risk urban youth. The re-developed risk 
assessment and the original risk assessment with re-weighted items had greater 
differentiation between moderate and high risk youth in these subgroups than did the 
other risk assessment versions. The re-developed risk assessment provided slightly 
greater differentiation than did the original risk assessment with re-weighted items. 

Additional considerations in evaluating risk assessment versions are face validity and 
reliability. That is, does the assessment appear to officers to measure risk and, given the same 
referral, would various officers complete a risk assessment in the same way for that referral? 
Feedback from officers indicate that many oppose the risk item that assesses parents’ criminality 
because they feel it unfairly influences the youth, and that it takes significant work to determine 
whether or not a parent has a criminal background. In re-developing the risk assessment, this 
item was eliminated because it has a weak association with recidivism relative to outcome 
measures. Therefore, officers may be more likely to accept the re-developed risk assessment as 
having face validity. Eliminating this item may also improve reliability in that officers have 
indicated how difficult it is to obtain parent criminal activity information. 

Results of this study indicate that the re-developed risk assessment and the original risk 
assessment with re-weighted items attain the best separation between risk levels. While both 
assessments achieved this regardless of the youth’s gender, ethnicity, or area of residence, the re- 
developed assessment achieved slightly greater separation between moderate and high risk youth 
for females and youth of non-white ethnicity. In addition, the re-developed assessment does not 
assess parents’ criminality, an item that officers have opposed in the past. 

The decision to modify the original risk assessment is, however, based both on research 
and policy. A number of policy issues affect risk assessment modifications: 

= Changes to the risk assessment would need to be made in BANNER (the statewide 


information system), which could be expensive and/or delay the implementation of 
BANNER; 
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= Changes to the risk assessment may also need to be made to J-TRAC, unless those 
counties would use the original risk assessment until BANNER implementation; and 


= Changes would need to be made to the manual (which may include changes to 
definitions), and those changes communicated to officers and other staff (through training 
or other means). 
Note, however, that all of the altered risk assessment versions reviewed in this report are 


similar to the original risk assessment (see Appendix A); which may ease the burden of 


modifications, particularly those related to manual changes and communicating changes to staff. 
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Appendix A 


Risk Assessment Forms 
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1. THE CURRENT MISSOURI JUVENILE RISK ASSESSMENT 




















Juvenile Name Parent Name Juvenile SS Number - - 
If juvenile has no SSN, use parent’s) 
Juvenile Date of Birth / / Juvenile ID# Race Gender M_ F 
Present Offense Code (list multiple offenses) ; , Juvenile Officer, 
Date Referral Received / / Date Form Completed / / County Circuit 


R1. Age at First Referral (record actual age ): 
Bs TL Otrcesish 28s tus edh coda se cdeccvontebsdetcd ostlvees godtvoagcdtvensade Moubeawscaveds toaenbed sbutee cogndvubed sssnieadsassenued su Sunde ap daibegcestbeds apuconeddbovvnde sossibed eburtuaeadtevedsoaveds coves 
b. 13,14, or 15 (circle actual age). 
c. 12 and under wo... eee 





R2. Prior Referrals (record actual number of referrals ) 
As SIN ONG ei fes accsnse anbilicsnea tunes xncceusecdeschnrycncatacesagedtyst an soa thngs totesscs Syst oars tose euade diteas be Guede ctu ohoeswis Weatiees de SNR ee Sen does ee Maat es eee es 0 


R3. Assault Referrals (record actual number of referrals __) 
a:..Noiprior or: present Tefertal for assaull ts $cc, scccses vie pasas Ses aabedets ns egenes Sosudee keseseg es tasecceat siete sesh osu eee aad reece 
b. One or more prior or present referrals for misdemeanor assault... 7 
c. One or more prior or present referrals for felomy assault ...........cccceecceeccesecssesecesecsecseesecsecasecseesecssecasecseeseceaecasecaeeecaecaaecseeeeeeaeeaeeneees 





R4. History of Placement 
a. No ‘prior out-of-home placement, ..nihnoe viet Aceh RAG ae Aen a Waa web Ra neuen aaa Aiea nos 0 
b. Prior out-of-home placement ......... ce eeseseesscsseseceseescseesecseesesecsaesecnevsccaesecsevsecsesesecsaecessesecsessecsesaeeecsessecseesecseaesecsaesecneveessesaeeeeaseeen 3 


R5. Peer Relationships 
a: Neuttal iifluence 3: sic. sesti sad eeee de evel ge catcaed secice saanetes ais vey dh cevelete ved esanceslausecessebidsds added Asedessctbvasdb caveaedeacdvevevestedevsssbeissbsbensdeaeretesaas 
b. Negative influence 
c. Strong negative influence 






R6. History of Child Abuse or Neglect 
a. No prior child abuse or neglect 
b. Prior child abuse or neglect ......... 
(petition filed or DFS finding of probable cause) 
Not Verified (score item from self-report) A 











R7. Substance Abuse 






a.. No‘aleohol:ordrug: problem’ jc: sayin eee aiieite Ba hea ae ee ee Mee igs eae he eee eee ee as 0 

b. Alcohol and/or drug abuse problem... eeecceseeceeeceteceeeeeeeeceeeeneees wl 

c. Severe Alcohol and/or drug abuse dependence ...........ecceceesssssceseeseesecesecsnecseeseesecaecssecseesecesecaaecseeeecaecssecseeseceaecaaecaeeseeaecaaecseeeeenaeenteess 3 
R8. School Behavior Problems 

a. No-oronlyaminor problems. oii ohne thas bade dee dedi bd teh Ha Gulsien dhe bea neha dhe Bint ies 0 

be Moderate. problems sascosis sevssrcasehetees coveted sates caguenaeiceveteat eaxeseoaacacashs geeaceustencevatunasenssnes cosutes aesucgnus caveauas acnanca guvateueamassaasoeeaeascacae hs auveneeetys 2. 

Ci Severe Problems. s.sccsscescvecsscccvccssecavestsescasetcs cavecedacsbeasdectvesdsuscescdse cuvecuceddechVestdesdaued oe cavecedacsbessdestdesddesdevedse duvecs dessbet dessPestdeceeseade cubeestesed 4 
R9. Parental Management Style 

A. Positive MaNaQeMENt .........ceceeceecceseeseessceecsecseesecesecssecseesecsecssecseesecesecasecsecsecscesecasecseceecaecasecseeseceaecasecseeseeaecaaecseeeceaecaaecseeeeeeeeaeeas 0 

b. Moderately ineffective management weil 

ib; Severely ineffective manag Cment ve. s2exseccavde sed sscecessssesbacs sonchuases vovies snaaaesugeskd axes aaeta dh gasueaabtsacbigs dovshaanes xovace saemmans vas ssaneneneatesedaeleats 3 





R10. Parents’ Criminal History 





a. No priorincarceratiOns es -sdavvae ade ities einiasiesteaianendiye tends cnane ayia aie eie deni eatenani dn dea wards ween a eden. 0 
Dic PIIOT INC ar COT at ONS pect rec cecse te teeta anc ceeh hes test ae cattery Se ee suc atiice Cosas ees toc vay Sass Dae Sesan nc oapcen Cece nae ae ae test olga hes State os cheeses eRe 3 
RISK LEVEL RISK SCORE 
High Risk +14 and above Motion to dismiss for cert. sustained MO 
Moderate Risk +8 to +13 Check action taken(check one): 
Low Risk 0 to +7 Ld Informal Adjustment C1 Adjudication 


DISCRETIONARY OVERRIDE 
If a discretionary override is made, circle yes, circle override risk level, and indicate reason. Risk level may be overridden one level higher. 











Yes No If yes, override risk level (circle one): Low Moderate High 

Discretionary Override Reason 
Supervisor’s Review/Approval of Discretionary Override: Date: / / 
DISPOSITION: SANCTIONS: (check all that apply) 


1) Allegation found true with petition 

LJ Allegation found not true with petition 

C1 Sustained motion to dismiss 

LI Informal adjustment conference 

1) Informal adjustment, no conference 

1) Transfer to another juvenile court 

1) Transfer to another facility 

(J Other transfer 

1) Referral rejected (check reason): 4 Allegation not true 
C) Insufficient evidence 











LI Restitution 

[1 Community Service 

LI Court Fees and Assessment 
1) Supervision 

1) Day Treatment 

LJ Intensive Supervision 

LI Out-of-Home Placement 

(1 Court Residential Placement 
1 Commitment to DYS 





(J Other sanctions not within matrix: 
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3. THE CURRENT MISSOURI JUVENILE RISK ASSESSMENT WITH RE-WEIGHTED ITEMS 


Juvenile Name Parent Name Juvenile SS Number - - 











If juvenile has no SSN, use parent’s) 











Juvenile Date of Birth / / Juvenile ID# Race Gender M _ F 
Present Offense Code (list multiple offenses) ; ; Juvenile Officer 
Date Referral Received / / Date Form Completed / / County Circuit 
Score 
R1. Age at First Referral (record actual age ): 
Ge 1 sce cabs asad de fede ta 26 sda cGosiad s5 Sdeted nebtvead aaGevelc a tevnes te Seebad ce ceeuda as aeted obsetcck ga bavbed soeWoudl 20 calead do cduedl anSeubedea wibeds te danske oWoedbandivaed assetedediscvend soetee Dans 2 
b. 13,14, or 15 (circle actual age). 


c. 12 and under ou... cee eeeeeeteeee 





R2. Prior Referrals (record actual number of referrals ) 
As SIN ONG ei fes accsnse anbslicsn eases unc ceussadeschrycncataceagedtystan soa tunas totesses Syst oars tose suede ditess be Gua ae cducGohoeswis Weadiees Se SNR cei Se Heese MS eah ee See 0 


R3. Assault Referrals (record actual number of referrals __) 
a:..Noiprior or: present Tefertal for assaull ts $3 ec, scccses vl pases Ses aanedete ne ege nes Socata kesesed es tasecaet she tetsee os eee eed 
b. One or more prior or present referrals for misdemeanor assault... 3 
c. One or more prior or present referrals for felomy assault ...........ccceecceeccesecseesecesecsecseesecesecasecseeecssecseceaecseeseeaecaaecaeeseeaecaaecseeeseeeeeaeeas 





R4. History of Placement 
a No ‘prior out-of-home placement, ..n.ci hoe veel ek Ae AAG ae Aen ae Waa web Ra aun eae Ai ene 0 
b. Prior out-of-home placement .......... cc eeesssessceseseceseeecseesecseesesecssesecnevsccaesecsevsccsesecscsaeccnaesessesecsesaeeecseesecsevsecseaesecsaesesnevaesieeaeeeeeaeeeen 1 


R5. Peer Relationships 
a: Neutral iifluence i.3: sci... sesti sad ceee devote ge cabcaed sedice sadavites asia ve sh cevelete ved oevanceslaatecessebidsds dbus Aedes ctbeasdb caveaedeaceevevestdevebeissosbendeaerenaeas 
b. Negative influence 
c. Strong negative influence 






R6. History of Child Abuse or Neglect 
a. No prior child abuse or neglect 
b. Prior child abuse or neglect ......... 
(petition filed or DFS finding of probable cause) 
Not Verified (score item from self-report) A 











R7. Substance Abuse 






a.. No‘aleohol:ordrug: problem’ ).ac sayin eee aiieita Bn keane ee eee ie eae Ge ee ee ee es 0 

b. Alcohol and/or drug abuse problem... eeecceseeceeeceteceeeeeeeeceeeeneees wl 

c. Severe Alcohol and/or drug abuse dependence .........ecceceesssssceseessesecesecececseeseesecsecesecseesecesecaaecseeeeecaecasecseesecaeceaecaeeeeeaecaaecaeeeeeeaeeseees 2 
R8. School Behavior Problems 

a. Novor.only minor problemseai.s isk heen. tase deh etneenihe Hatiieaen deed ouliiadaied hn eu adage hed ieks -1 

be Moderate. problems sascosis sessvcasehetees coveted sates cagensbiacoveteat eavessoaacacasts geeaceuttencevatuuasenssnes cosutes aesucgnus caveauas acnanca guvateueamassabsoeeaeascacae hs aovekeuers 0 

Ci Severe Problems... scsscccscescvecessccvecssecavestsescasence cavecedacsbeas dost vesdsuscescdse cavecsceddechtestdesdaded oe cavece dacsbeesdectdesdde sdevedse duvecs tessbet desePestdaceesedde cubecseteses 1 
R9. Parental Management Style 

A. Positive ManageMENt ......... ce eeceeeceeccesecseesscesecsecseeeecesecssecseesecesecsaecseeseceaecssecseesecescesecsaecseeeecaecasecseeseceaecasecseeecaecaaecaeeeeeaecaaecseeeeeeeeaeess 0 

b. Moderately ineffective management wl 

ib; “Severely ineffective manag Cment sve... xseccavdv aed ss cecessusesbacs danchuases covied soausa esugeakd ais cascade sageenab tsa cbigs douctaanes covace seemans vas usanennetedae aed 2 





R10. Parents’ Criminal History 





as. NO prioranCarceratiOns ss sess-ceedeviviseeycalances bast deved me niegaiesteeatenandeneisneiy cand eueoec nance iyi aio ade ent eatananie nde arses a cee aed aed. 0 
Dx PHIOK INCarCeratlOns esas cae ec cect oaccuct cen ested cotietnn Sues tetea crates cena oan Ses Guay cess aaa Temgeth «Sapien dace nde cu ae testo ge oe phic, Sette os reeset eee 1 
RISK LEVEL RISK SCORE 
High Risk +8 and above Motion to dismiss for cert. sustained MO 
Moderate Risk +1 to +7 Check action taken(check one): 
Low Risk -3 to 0 (1 Informal Adjustment LI Adjudication 


DISCRETIONARY OVERRIDE 
If a discretionary override is made, circle yes, circle override risk level, and indicate reason. Risk level may be overridden one level higher. 











Yes No If yes, override risk level (circle one): Low Moderate High 

Discretionary Override Reason 
Supervisor’s Review/Approval of Discretionary Override: Date: / / 
DISPOSITION: SANCTIONS: (check all that apply) 


1) Allegation found true with petition 

LJ Allegation found not true with petition 

C1 Sustained motion to dismiss 

LI Informal adjustment conference 

1) Informal adjustment, no conference 

1) Transfer to another juvenile court 

1) Transfer to another facility 

(J Other transfer 

1) Referral rejected (check reason): 4 Allegation not true 
C) Insufficient evidence 











LI Restitution 

[1 Community Service 

LI Court Fees and Assessment 

1) Supervision 

1) Day Treatment 

LJ Intensive Supervision 

LI Out-of-Home Placement 

(1 Court Residential Placement 

1 Commitment to DYS 

(J Other sanctions not within matrix: 
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4. THE RE-DEVELOPED JUVENILE RISK ASSESSMENT 

















Juvenile Name Parent Name Juvenile SS Number. - - 
If juvenile has no SSN, use parent's) 
Juvenile Date of Birth i / Juvenile ID# Race Gender M_ F 
Present Offense Code (list multiple offenses) : : Juvenile Officer. 
Date Referral Received / / Date Form Completed / / County Circuit 


Score 
R1. Age at First Referral (record actual age ): 
BLO sesicis ee Se covetvuauseat oatvachbath aa vaesecaeesses doatea onsuwevany sodeka beans sus pads daelunss sodass te degsabaoasvadhaoaaeatu as sovacs dadunadteeieaap duceuisbunacdees doaeeaaneaneyaes shsess Sapestaandoss 
b. 13,14, or 15 (circle actual age) : 
Co AD ANGANG SL salsa sei Seeacskesessontesieadeslastitea visatedoseuesaanebia losegsaare aeusnsaluszeavesee5 cs soresbande tousosendedies upte dedtedeSealosanie bendnevto rp sedbeats jedbeaseantend seguass 





R2. Prior Referrals (record actual number of referrals ) 
Asc NONC ss cccssentsitevesseavds Seseatea nes veausarsetbasesversurs) curs dg’ssaeneuvenncgeeedesegntbatessesteh gatesa shesseabecdesneesseepune iastaseanedtoepsadesadhcvacsbanddecgusessestaaberdepedbeunsseetsnses 
b. One or two prior referrals ........ 
c. Three or more prior referrals 





R3. Assault Referrals (record actual number of referrals __) 
a. No prior or present referral for assault 2.0.0.0... es sceesscneesecsesseeccseesecueescssesecneesecsesaccecsaesecnevsccassecsesassccsavsecsesesaesaeeecsassesneusessaaseecnaeees 0 
b. One or more prior or present referrals for misdemeanor or felony assault.........cceeeeccesecsceesceeecesecsecseeesceseccsecaeeeeceseceaecaeeeseeeeeaeeneeeeees 1 


R4. History of Placement 
a. No prior out-of-home placement... cvcsccssdascccscovccdssccesscectvesacudcs cavecececeucescecuteed dosdesetcncubece dacabecs dacsaicadesdescacecauees dasstent dessdusddedenecssecu¥essdeses 0 
bz Prioroutzof-home: placement i. cece. sec sesecesexcevaeervacnsesanssssusienseceeaapss sees vanset svsdawandes dures sesdsdensesins vt ausdenasces dupes saestgndes daveb evedouaniesiNea sea suiedes 1 


R5. Peer Relationships 
a. Neutral influence...... 
b. Negative influence 
c. Strong negative influence 












R6. History of Child Abuse or Neglect 
a. No prior child abuse or neglect 
b:; Prior:child'abuse:or Neglect pias. cess swsptsceceayseesvecs ye Suescpatsysied ch evsd iti caveceaidietepecbabehonayusauyeag Michans acto bagisa sailed eveaera eevee hed 
(petition filed or DFS finding of probable cause) 
Not Verified (score item from self-report) A 





R7. Substance Abuse 
a. No alcohol or drug problem ...........ccccceessesscsseeseesecescesecsceeseeseceaecasecseesecsaecasecsecseceaecaseeseeseceaceaaeeseeeeseasente 0 
b. Alcohol and/or drug abuse problem................... me 
c. Severe Alcohol and/or drug abuse dependence... .ecccceesssscceseeseesscesecesecseeseeecaecasecseesecesecaaecaeeseecsecaaecseesecsaecaaecseeeceaecaaecaeeeeeeaeeaeens 2 






R8. School Behavior Problems 
a: Noor only: minor problems: is:.iece siete Adsiwn Lada teaiatinsla ite and aetna Ahr dcige ei ae 
b. Moderate or severe problems 





R9. Parental Management Style 





ay Positive ManageMeNt 2:2. civecs ssccvecssecuvesdansdauvegs cavicesaceuess coisa oy sdesed cn cuveas dasshens du ceDuesivd on cueassacevees Gosiiisad on sdeues de codes secublees de Seduadsateesdde cuvess dase 0 
bz. Ineffective managements. ccczence sis ssvays cesceuivesaxgs vous ies ssnsbetetsnaniend vest ens sueneia inva visi ieee supeibes sys ueelewsi ses Gusto aeathendesd a aceeeeceeiaveae eae 1 
RISK SCORE 
RISK LEVEL 
High Risk +8 and above Motion to dismiss for cert. sustained MO 
Moderate Risk +2 to +7 Check action taken(check one): 
Low Risk -2to+1 Li Informal Adjustment LI Adjudication 


DISCRETIONARY OVERRIDE 
If a discretionary override is made, circle yes, circle override risk level, and indicate reason. Risk level may be overridden one level higher. 














Yes No If yes, override risk level (circle one): Low Moderate High 

Discretionary Override Reason. 
Supervisor’s Review/Approval of Discretionary Override: Date: / / 
DISPOSITION: SANCTIONS: (check all that apply) 


1) Allegation found true with petition 

Ly Allegation found not true with petition 

L) Sustained motion to dismiss 

1) Informal adjustment conference 

1) Informal adjustment, no conference 

1) Transfer to another juvenile court 

1) Transfer to another facility 

(J Other transfer 

1) Referral rejected (check reason): 4 Allegation not true 
1 Insufficient evidence 











LI Restitution 

[1 Community Service 

LJ Court Fees and Assessment 
1) Supervision 

1) Day Treatment 

Intensive Supervision 

(1 Out-of-Home Placement 

(1 Court Residential Placement 
1 Commitment to DYS 





(1 Other sanctions not within matrix: 
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Appendix B 


Item Analysis 
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Table B1 


Item Analysis: Current Risk Assessment 






































































Sample Subsequent Referral Subsequent Adjudication 

Risk Assessment Item N P value N P value 
Total Sample 2,911 100.0% 994 34.1% 249 8.6% 
Age at First Referral 

16 12.5% 

13,14 or 15 54.5% 

12 and under 33.0% 
Prior Referrals 

None 49.7% 

One or more 50.3% 
Assault Referrals (Prior or Present) 

No prior or present assault referral 68.4% 

One or more misdemeanor assault 24.2% 

One or more felony assault 7.5% 
History of Placement 

No prior out-of-home 2,290 78.7% 696 30.4% 173 7.6% 

Prior out-of-home | 621 =| 21.3% 298 | 48.0% | 76 | 12.2% | 
Peer Relationships 

Neutral influence 50.3% 

Negative influence 42.6% 

Strong negative influence 7.1% 
History of Child Abuse/ Neglect 

No prior CA/N 84.0% 

Prior CA/N history 467 16.0% 
Substance Abuse 

No problem 71.9% 

Moderate problem 25.6% 

Severe dependence 2.5% 
School Behavior Problems 

No or minor problems 49.0% 

Moderate problems 39.5% 

Severe problems 11.5% 
Parental Management Style 

Positive management 57.6% 

Moderately ineffective management 33.8% 

Severely ineffective management 249 8.6% 
Parents’ Criminal History 

No prior incarceration 77.6% 

Prior incarceration 651 22.4% 587 
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Table B2 


Item Analysis: Risk Assessment with Re-Weighted Items 
















Sample 





Subsequent Referral 








Subsequent Adjudication 














12 and under 

Prior Referrals 
None 
One or more 

Assault Referrals (Prior or Present) 
No prior or present assault referral 
One or more misdemeanor assault 
One or more felony assault 

History of Placement 

No prior out-of-home 

Prior out-of-home 

Peer Relationships 

Neutral influence 

Negative influence 

Strong negative influence 

History of Child Abuse/ Neglect 

No prior CA/N 

Prior CA/N history 

Substance Abuse 

No problem 

Moderate problem 

Severe dependence 

School Behavior Problems 

No or minor problems 

Moderate problems 
Severe problems 

Parental Management Style 
Positive management 
Moderately ineffective management 
Severely ineffective management 

Parents’ Criminal History 
No prior incarceration 
Prior incarceration 














7.5% 


2,290 
| 621 


1.1% 


467 


2.5% 


249 


651 


33.0% 41.6% 


49.7% 
50.3% 


68.4% 
24.2% 





78.7% 696 30.4% 

| 21.3% 298 | 48.0% | 
50.3% 
42.6% 


84.0% 
16.0% 


71.9% 
25.6% 


49.0% 
39.5% 
11.5% 


57.6% 
33.8% 
8.6% 


77.6% 
22.4% 587 














Risk Assessment Item N P value N P value 
Total Sample 2,911 100.0% 1032 35.5% 254 8.7% 
Age at First Referral 

16 12.5% 

13,14 or 15 54.5% 








7.6% 
| 12.2% | 
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Table B3 


Item Analysis: Re-Developed Risk Assessment 















Sample 









Subsequent Referral 


Subsequent Adjudication 













Relationships 
Neutral influence 
Negative influence 
Strong negative influence 
History of Child Abuse/ Neglect 
No prior CA/N 
Prior CA/N history 
Substance Abuse 
No problem 
Moderate problem 
Severe dependence 
School Behavior Problems 
No or minor problems 
Moderate or severe problems 
Parental Management Style 
Positive management 
Ineffective management 





Peer 



























2,094 


1,233 








71.9% 
25.6% 
2.5% 


49.0% 
51.0% 


57.6% 
42.4% 









































Risk Assessment Item N P value N P value 
Total Sample 2,911 100.0% 994 34.1 249 8.6 
Age at First Referral 

16 12.5% 

13,14 or 15 54.5% 

12 and under 33.0% 
Prior Referrals 

None 49.7% 

One or two 49.8% 

Three or more 0.5% 
Assault Referrals (Prior or Present) 

No prior or present assault referral 68.4% 

One or more assault referral 31.6% 
History of Placement 

No prior out-of-home 78.7% 

Prior out-of-home 621 21.3% 
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Appendix C 


Suggested Risk Reassessment 
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Risk Reassessment 

The purpose of risk reassessment is to measure changes in a youth’s risk of future 
delinquency based upon response to services and other changes (such as a change in living 
arrangement). A risk reassessment may be completed periodically such as every three months, at 
judicial review, or when a significant change occurs with the youth that may affect risk level. 

The risk reassessment scale combines items from the original risk assessment tool with 
additional items that evaluate a youth’s progress toward case plan goals. The risk reassessment 
tool is in part consensus-based in that many items that assess change over time are not based 
upon research. Some items, however, are carried over from the initial risk assessment and are 
actuarial. 

The Risk Assessment Committee chose to adopt the original risk assessment with re- 
weighted items and revised cut points. A suggested risk reassessment to be used in conjunction 


with that risk assessment is shown on the following page. 
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SUGGESTED RISK REASSESSMENT 




















Juvenile Name Parent Name Juvenile SS Number - - 
If juvenile has no SSN, use parent’s) 
Juvenile Date of Birth / / Juvenile ID# Race Gender M_ F 
Present Offense Code (list multiple offenses) : Juvenile Officer 
Date Referral Received / / Date Form Completed / / County Circuit 


Score 
Fill in questions RE1 — RE3 from the initial risk assessment completed at time of disposition. 
RE1. Age at First Referral (record actual age ): 


b. 13,14, or 15 (circle actual age).... we 
AO aI TIL ese ete hate ecetlins as ES htce AM lan IAM nie ine Nate onde ia: lee Neal NN 





RE2. Prior Referrals (record actual number of referrals __) 
MeN OIG sas dey sh ys cekesess cape a asek conte den cay ates sa sae gostsnsaedss datos bas caasagndayaatieouassgy aie sSekeespancdhiay to basousssa dapat atensasachdvpabaves ona cigedspesQusaaabenivesekeevaanesy 
b. One or more prior referrals 





RE3. Assault Referrals (record actual number of referrals ) 
a. No prior or present referral for assault ..........ccecccesecsseesceseceseeseesecesecasecsecseeseceaecaseeseeseceaecaeeeseeseceaecseeeseeseceaecaaeeseeseceaecaeeeseeeceaeeneensens 
b. One or more prior or present referrals for misdemeanor assault 
c. One or more prior or present referrals for felony assault ................... 





When scoring the following items, use updated information since the initial risk assessment or most recent risk reassessment. 


RE4. Non-delinquent findings (technicals) by court or Juvenile Parole Board 
AP INOME, 3. sdancctasetestesecsuvavest souchses neuer’ vyde pce ute tea dusad ua cuted oy sauers se wxw Oe deataneyss cede 3a sau ray Sheeyete eesk vera osu aaa Bed seas oe auss sok Ces stoeg tate Neda sate caeat ee 
b. One or more 





RES. Current Peer Relationships 
a: Neutral ntluence iiss. sc secaieses oo fectsonta Gthibtesenba thawte der ctaarienad acts node dane ater acne nahh leat 
b. Negative influence............... 

c. Strong negative influence 





RE6. Current Substance Use 


BA INO's. sssiiasestcss tase siesteanastantn ss seus canes Coser ses eb pnp Case Wahen oho asses Sout Gon Tas uated wahes taveasa snes hye Geyneos lobe as bantaped viet ebnestevessssen’ wauenbaveesihees has tavess 0 

Ce LYOS  socsuateued joetatgese shes saswna oo estevsca soxedabsacdhaey devmaasyene aia aa saree pseowavech decttys soveashuce dns aha ales eva oh dustacssusanhaee daautaasunsahe a duedea phtavoueednonvasiesestaas etek 2 
RE7. Runaways from Home or Community-Based Placement 

a NO ai.iishat diets cesanguaioatin ie aie iter ieeeoaitaiinn Beeae oily aA and aioe sual aa ae anevada uum N -1 

Be VS esc sesicses sail csesdvea ess sodediedssscsetesedacesavpevsealodesvetanapiseoedndusosoven iesdsdonasdsdupaasventgagweod sy tvsasday veeuspgesieatesaabedspdaigud poles b¥o aastosvengeb onde’ +1 





RE8. Current School (__) or Employment (__) Problems (check applicable status) 
a: No ‘problems;.or problems: very minor ssic....5 su wives died Reta tna Gedieed wl he Nea Ah AERA Lee ae ea 
b. Some attendance problems requiring conferences at school, short-term suspension, job loss .. 
c. Sanctioned at school or does not enroll; fails to seek and maintain employment... cee ecceeccesecseeeeceecceececseeeeceseceaecseeeeeeaeenseens 











RE9. Program Adjustment on Supervision 









a. Satisfactory — no behavior problems ............cccceeccessessesseeeecesecseeesceseceaecnsescesseseceaecaeeeseseceaecssesseeseceaecaseeseeseceaecaeeeseesecaecaseeseeeseaeente -1 
b. Fair—some problems; no major infractions; no revocations filed 0 
c. Poor — chronic adjustment problems; major infractions requiring delinquency action or violation of probation petition filed .....+2 
RISK SCORE = 
SCORED RISK LEVEL 

High Risk +8 and above 

Moderate Risk +1 to +7 

Low Risk -3 to 0 


DISCRETIONARY OVERRIDE 
If a discretionary override is made, circle yes, circle override risk level, and indicate reason. Risk level may be overridden one level higher or 
lower. 


Yes No If yes, override risk level (circle one): Low Moderate High 
Discretionary Override Reason 














Supervisor’s Review/Approval of Discretionary Override: Date: / / 
FINAL RISK LEVEL 1. Low 2. Moderate 3. High 
REASSESSMENT TYPE: 1. Routine 2. Change risk classification 3. Extension of probation 
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Appendix D 


Law and Status Outcomes 
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LAW OUTCOMES 


The following tables review law outcomes by the version of the risk assessment for the 
overall sample (that is, outcomes involving only law offenses). Table D1 shows that when the 
original risk assessment is applied, 15.3% of youth classified as low risk had a subsequent law 
referral, while 43.9% of high risk youth classified had a subsequent law referral. The altered risk 


assessment versions showed greater separation between youth classified as low risk and high 


risk, as well as between consecutive risk levels. 








Table D1 


Findings for Subsequent Law Referral 
by Risk Assessment Version 





Subsequent Law Referral 













































































Total N N % 

Total Sample 2,911 777 26.7% 
1. Original Risk Assessment 

Low Risk 1,429 218 15.3% 

Moderate Risk 888 298 33.6% 

High Risk 594 261 43.9% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 109 11.7% 

Moderate Risk 1,664 513 30.8% 

High Risk 312 155 44.7% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 77 9.7% 

Moderate Risk 1,790 532 29.7% 

High Risk 329 168 51.1% 
4. Re-Developed Risk Assessment 

Low Risk 776 77 9.98% 

Moderate Risk 1,608 447 27.8% 

High Risk 527 253 48.0% 








Table D2 similarly compares classification findings when the outcome is law referrals 


accepted for investigation. 


developed risk assessment showed a greater difference in re-referral rates between low and high 


The original risk assessment with re-weighted items and the re- 


risk youth when compared to those the other two assessment versions. 
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D1 



















































































Table D2 
Findings for Subsequent Accepted Referral 
by Risk Assessment Version 
Subsequent Accepted Law Referral 
Total N N % 

Total Sample 2,911 551 18.9% 
1. Original Risk Assessment 

Low Risk 1,429 167 11.7% 

Moderate Risk 888 207 23.3% 

High Risk 594 177 29.8% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 90 9.6% 

Moderate Risk 1,664 358 21.5% 

High Risk 312 103 33.0% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 60 7.6% 

Moderate Risk 1,790 384 21.5% 

High Risk 329 107 32.5% 
4. Re-Developed Risk Assessment 

Low Risk 776 59 7.6% 

Moderate Risk 1,608 331 20.6% 

High Risk 527 161 30.6% 


























The re-developed risk assessment and the original risk assessment with re-weighted items 
also achieved greater separation between youth classified as low risk and high risk when the 


observed outcome is subsequent petitioned law referrals (see Table D3). 








Table D3 


Findings for Subsequent Petitioned Law Referral 
by Risk Assessment Version 






























































Subsequent Petitioned Law Referral 
Total N N % 

Total Sample 2,911 270 9.3% 
1. Original Risk Assessment 

Low Risk 1,429 53 3.7% 

Moderate Risk 888 105 11.8% 

High Risk 594 112 18.9% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 27 2.9% 

Moderate Risk 1,664 177 10.6% 

High Risk 312 66 21.2% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 14 1.8% 

Moderate Risk 1,790 182 10.2% 

High Risk 329 74 22.5% 
4. Re-Developed Risk Assessment 

Low Risk 776 14 1.8% 

Moderate Risk 1,608 146 9.1% 

High Risk 527 110 20.9% 
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Risk Assessment Classification Findings for Law Outcomes by Subgroups 

Table D4 presents findings for the five risk assessment versions by gender when the 
outcome is subsequent law referral (accepted or not accepted). Overall, males were nearly twice 
as likely (96% more likely) to have a subsequent law referral than were females (base rates are 
31.4% and 16.0%, respectively). Despite the significant difference in base rates, all four risk 
assessment versions classified both male and female youth such that an increase in the risk level 
corresponded to an increase in the re-referral rate for law offenses. The original assessment with 


revised cut points produced the greatest distinction between moderate risk males and high risk 



















































































females. 
Table D4 
Findings for Subsequent Law Referral 
by Youth Gender 
Male Female 
Subsequent Law Subsequent Law 
Total Referral Total Referral 
N N % N N % 

Total Sample 2,018 634 31.4% 893 143 16.0% 
1. Original Risk Assessment 

Low Risk 927 173 18.7% 502 45 9.0% 

Moderate Risk 659 245 37.2% 229 53 23.1% 

High Risk 432 216 50.0% 162 45 27.8% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 592 85 9.0% 343 24 7.0% 

Moderate Risk 1,190 416 23.1% 474 97 20.5% 

High Risk 236 133 27.8% 76 22 28.9% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 497 59 11.9% 295 18 6.1% 

Moderate Risk 1,272 433 34.0% 518 99 19.1% 

High Risk 249 142 57.0% 80 26 32.5% 
4. Re-Developed Risk Assessment 

Low Risk 484 60 12.4% 292 17 5.8% 

Moderate Risk 1,140 366 32.1% 468 81 17.3% 

High Risk 394 208 52.8% 133 45 33.8% 
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Table DS shows that when comparing risk assessment performance by ethnicity for the 
outcome subsequent law referral, the original risk assessment did not differentiate between 


moderate and high risk non-white youth as well as the other risk assessment versions did. 

























































































Table D5 
Findings for Subsequent Law Referral 
by Youth Ethnicity 
White Non-White 
Subsequent Law Subsequent Law 
Total Referral Total Referral 
N N % N N % 

Total Sample 1,935 409 21.1% 968 366 37.8% 
1. Original Risk Assessment 

Low Risk 1,124 157 14.0% 299 61 29.8% 

Moderate Risk 543 160 29.5% 344 137 51.5% 

High Risk 268 92 34.3% 325 168 63.4% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 777 80 10.3% 152 29 26.3% 

Moderate Risk 1,038 280 27.0% 625 232 48.3% 

High Risk 120 49 40.8% 191 105 68.1% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 677 59 8.7% 111 18 21.6% 

Moderate Risk 1,132 299 26.4% 655 232 47.2% 

High Risk 126 51 40.5% 202 116 68.8% 
4. Re-Developed Risk Assessment 

Low Risk 654 58 8.9% 118 19 22.0% 

Moderate Risk 1,047 262 25.0% 559 185 44.9% 

High Risk 234 89 38.0% 291 162 67.0% 
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STATUS OFFENSES 
Table D6 shows that the risk assessment versions also classified youth such that an 


increase in risk level corresponded to an increase in the rate of subsequent status referrals. 











Table D6 


Findings for Subsequent Status Referral 
by Risk Assessment Version 










































































Subsequent Status Referral 
Total N N % 

Total Sample 2,911 316 10.9% 
1. Original Risk Assessment 

Low Risk 1,429 105 7.3% 

Moderate Risk 888 112 12.6% 

High Risk 594 99 16.7% 
2. Original Risk Assessment with Revised Cut Points 

Low Risk 935 50 5.3% 

Moderate Risk 1,664 216 13.0% 

High Risk 312 50 16.0% 
3. Original Risk Assessment with Re-Weighted Items 

Low Risk 792 32 4.0% 

Moderate Risk 1,790 232 13.0% 

High Risk 329 52 15.8% 
4. Re-Developed Risk Assessment 

Low Risk 776 30 3.9% 

Moderate Risk 1,608 197 12.3% 

High Risk 527 89 16.9% 
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